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ABSTRACT 

The Connecticut Mastery Test testing program assesses 
essential skills in mathematics and language arts, including 
listening, reading, and writing, for grades 4, 6, and 8* For grade 8, 
student achievement is measured and reported in relation to specific 
learning objectives (criterion-referenced) that students can 
reasonably be expected to have mastered by the end of grade ?• In 
1991, eighth graders mastered an average of 25*8 or the 36 
mathematics objectives tested, a slight improvement over the 1990 
result* A total of 87*8 percent scored at or above the remedial 
standard, equaling last year's figure, A total of 38*1 percent scored 
at or above the mathematics goal, an improvement over 1990* s 37 
percent. In language arts, the mastery of 8*3 of 11 objectives 
represented a slight decline from 1990, In writing, eighth graders 
showed improvement in meeting objectives and scoring .ibove the 
writing goal, but there was a slight decline in students scoring 
above the remedial standard, from 93*2 percent in 1990 to 92*5 
percent in 1991* Eighth graders averaged 63 units on the Degrees of 
Reading Power, equaling 1990*s figure* Small improvements were 
registered in the numbers of students scoring above the remedial 
standard and the reading goal* Fourteen charts present test results 
and comparative information from 1986 through 1991. Twelve appendixes 
provide supplemental information on testing and scoring procedures* 
(SLD) 
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LEGISLATIVE BACKGROUND 



In June 1984, the General Assembly of the State of Connecticut amended Section 
10-14 m-r of the Connecticut General Statutes, an act concerning Education 
Evaluation and Remedial Assistance (EERA)^ This law provides that: 

0 By May 1, 1985, each local or regional board of education shall have 
developed and submitted for State Board of Education approval, a new 
plan of educational evaluation and remedial assistance. Each plan 
had to address the following: 

0 the use of student assessment results for instructional 
improvement; 

0 the identification of individual students in need of remedial 
assistance in language arts/reading and mathematics; 

0 the provision of remedial assistance to students with identified 
needs; and 

0 the evaluation of the effectiveness of the Instructional 
programs in language arts/reading and mathematics- 

0 The State Board of Education shall administer an annual statewide 

mastery test in language arts/reading and mathematics to all fourth--, 
sixth- and eighth-grade students, with the following exceptions: 

0 Special Education students who are excluded by a Planning and 
Placement Team (PPT) decision; 

0 students who have been enrolled in an "English as a Second 
Language" program for two years or less; or 

0 students enrolled in a Bilingual Program (as defined in Section 
10-17e of the Connecticut General Statutes) for two years or 
less* 

0 Each student who scores below the statewide remedial standard on one 
or more parts of the eighth-grade mastery examination shall be 
retested* These students shall be retested annually, using the 
eighth-grade mastery test, only in the deficient area(s) until such 
students score at or above the statewide remedial standard(s). 

0 Biennially, each local or regional board of education shall submit to 
the State Board of Education a report which includes indicators of 
student achievement and instructional improvement. 

0 On a regularly scheduled basis, the State Board of Education shall 
complete field assessments of the implementation of local EERA plans. 
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0 On an annual basis, test results and low income data shall be used to 
determine the distribution of available state funds to support 
remedial assistance programs. 



The purpose of this report is to provide an overview and summary of the 
implementation of the eighth-grade Connecticut Mastery Test, The mastery test 
assesses how well each student is performing on those skills identified by 
content experts and practicing educators as important for students entering 
eighth grade to have mastered. 



FOREWORD 



The Connecticut Mastery Test is a critical element in Connecticut's agenda to 
attain educational equity and excellence. The testing program assesses 
essential skills in mathematics and language arts, including listening, 
reading and writing, for grades four, six and eight students. Student 
achievement is measured and reported in relation to specific learning 
objectives that students reasonably can be expected to have mastered by the 
end of grades three, five and seven. 

The Connecticut Mastery Test provides valuable educational information which 
can be used to improve instruction and elevate the achievement of 
Connecticut's students. The test results are reported in a manner that 
identifies how well each student is succeeding in relation to clearly defined 
and meaningful standards. It is our hope that educators throughout the state 
use the results as a tool to gain better understanding of the learning 
occurring in our classrooms and the ways to increase learning in the future. 

Connecticut is committed to an annual cycle of assessment in order to promote: 

0 the monitoring of individual student achievement; 

0 the evaluation of instructional program effectiveness; 

0 educational goal setting; and 

0 remedial assistance program improvement. 

An examination of the results since 1985 reveals many signs of steady, 
incremental improvement. The general improvement since the start of the 
program is quite impressive in some areas. Yet the many Connecticut educators 
who heloed to build the program had the foresight to include some very 
demanding content and standards. Student performance in relation to these 
expectations reveals that mi:ch remains to be done. 

As you examine these results, it is our hope that the many stories they tell 
will prove useful and informative. Department staff are available to 
facilitate the interpretations and application of these test scores. 

Peter Behuniak 
Acting Chief 

Bureau of Evaluation and StudenL Assessment 





OVERVIEW OF THE MASTERY TESTING PROGRAM 



In the spring of 1984, the Connecticut General Assembly amended the Education 
Evaluation and Remedial Assistance (EERA) legislation to authorize the 
creation of mastery tests in the basic skill areas of mathematics and language 
arts, including listening, reading and writing skills. The tests were to be 
established for grades four, six and eight* 

The goals of the mastery testing program are: 

0 earlier identification of students needing remedial education; 

0 testing a more comprehensive range of academic skills; 

0 setting high expectations and standards for student achievement; 

0 more useful test achievement information about students, schools and 
di stricts; 

0 improved assessment of suitable equal educational opportunities; and 

0 continual monitoring of students in grades four, six and eight. 

The type of test that best addresses these goals is a criterion-referenced 
test. Criterion-referenced tests are designed to assess the specific skill 
levels of students. Such tests usually cover relatively small units of 
content. Their scores have meaning in terms of what each student knows or can 
do. Test results are used to Identify the areas of strengths and weaknesses 
of each student. 



MASTERY TEST CONTENT 

The CMT is designed to assess essential language arts/reading, writing and 
mathematics skills that can reasonably be expected to be mastered by most 
students by the end of the third, fifth and seventh grades. The specific 
skills to be tested within these content areas were identified by committees 
of educators from throughout the state. In addition, surveys were sent to 
many teachers, administrators and parents to determine the appropriateness of 
these skills for the Mastery Test. A complete description of the procedures 
used in the development of the eighth-grade CMT can be found in Appendix A 
(p. 33). 

Mathematics 

The Mathematics Advisory Committee recommended a grade eight mathematics test 
that assessed thirty-six (36) specific objectives in four domains: (1) 
Conceptual Understanding; (2) Computational Skills; (3) Problem 
Solving/Applications; and (4) Measurement /Geometry. There are four test items 
per objective for a total of 144 items on the mathematics test. A detailed 
list of domains and objectives is given in Appendix B (p. 37). 



Language Arts 



The Language Arts Advisory Committee recommended a Ill-item grade eight 
language arts test that covers two domains: Reading/Listening and 
Writing/Study Skills. Eleven (11) objectives were recommended by the 
Language Arts Advisory Committee, 

The general content area of Reading/Listening consisted of narrative, 
expository and persuasive passages on a variety of topics measuring a 
student's ability in: (1) Literal Comprehension; (2) Inferential 
Comprehension; and (3) Evaluative Comprehension. Audiotapes were used to 
assess students* listening comprehension ability in: (1) Literal Comprehension 
and (2) Inferential and Evaluative Comprehension. The Degrees of Reading 
Power (DRP) test was also used to assess reading. The DRP test included 
eleven (11) passages and seventy-seven (77) test items. It was designed to 
measure a student's ability to understand nonfiction English prose at 
different levels of reading difficulty. 

The general content area of Writing/Study Skills consisted of three 
components. First, there was a writing sample for direct, holistic assessment 
of student writing. Each student was asked to write a composition on a 
designated topic. Writing was then judged on a student's demonstrated ability 
to convey information in a coherent and organized fashion. Second, the 
mechanics of good writing, which was defined as (1) Capitalization and 
Punctuation, (2) Spelling, (3) Agreement and (4) Tone, were assessed in a 
multiple-choice format. Third, Study Skills were assessed through Locating 
Information and Note--taking/Outl ining. Locating Information (Schedules, Maps, 
Index and Reference Use), measured a student's ability to find and use 
information from the sources listed. Note-taking and Outlining tested a 
student's ability to take notes and report information as well as complete 
missing outline information, A detailed list with objectives and number of 
items per objective is given in Appendix C (p. 41). 



FUTURE DEVELOPMENT 

The Connecticut State Department of Education (CSDE), in conjunction with 
content consultants and various CMT advisory committees, has begun the 
development of the second generation of the CMT. The current CMT is under 
review to determine which skills are appropriate for inclusion the new 
test. In addition, new content areas and other forms of assessment techniques 
(e.g., performance assessment and short-answer questions) are being 
considered. It is anticipated that the second generation CMT will be 
administered for the first time statewide in the fall of 1993. Items for this 
set of exams were piloted in the fall of 1991 and will be followed by a second 
pilot in the fall of 1992. 
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SETTING MASTERY STANDARDS BY OBJECTIVE 



The essence of the Connecticut Mastery Test (CMT) is the establishment of a 
specific mastery standard against which each student's knowledge and competency 
on each objective can be compared. The mastery test incorporates appropriate and 
challenging expectations for Connecticut public school students. The goal of the 
CMT Program is for each student to achieve mastery of all objectives. The 
objectives being tested were identified as appropriate and reasonable for 
students at each of the grades tested. These tests are designed to measure a 
student's performance on these specific objectives. 

The process of establishing the mastery standards by objective used a statistical 
method that required two decisions to be utilized. The first decision defined a 
student who mastered a particular skill as one who had a 95% chance of correctly 
answering each item within the objective. The second decision was that the 
specific standard for each objective would identify 99% of the students who 
mastered the skill. By applying the two decision rules stated above to a 
binomial distribution table, mastery standards were established for the 36 
mathematics objectives and the 11 language arts objectives. 

The mastery standards are as follows: 

0 In mathematics, for each of the 36 objectives, a student must answer 
correctly at least 3 out of 4 items. 

0 In language arts, for the 11 multiple-choice objectives with varying 
numbers of items, a student must answer correctly the following numbers 



of items: 



WRITING MECHANICS 



# Items Correct 
For Mastery 



(1) Capitalization & Punctuation 


9 


out 


of 


12 


(2) Spelling 


6 


out 


of 


8 


(3) Agreement 


11 


out 


of 


15 


(4) Tone 


3 


out 


of 


4 


STUDY SKILLS 










(5) Locating Information 


9 


out 


of 


12 


(6) Note-taking and Outlining 


3 


out 


of 


4 


LISTENING COMPREHENSION 










(7) Literal 


3 


out 


of 


4 


(8) Inferential and Evaluative 


12 


out 


of 


16 


READING COMPREHENSION 










(9) Literal 


6 


out 


of 


8 


(10) Inferential 


10 


out 


of 


14 


(11) Evaluative 


10 


out 


of 


14 



No mastery standards were set for the two holistic language arts measures, 
neither the Degrees of Reading Power (DRP) test nor the Writing Sample, since 
these measures are not composed of objectives on which mastery could be assessed, 
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SETTING REMEDIAL (GRANT) STANDARDS 



In addition to mastery standards, Section 10-14 m-r of the Connecticut General 
Statutes requires that the Connecticut State Board of Education establish 
statewide standards for remedial assistance in order to meet two 
responsibilities: 

0 to identify and monitor the progress of students in need of remedial 
assistance in language arts/reading and mathematics as part of the 
EERA field assessments; and 

0 to distribute EERA funds based on the number of needy students 

statewide, as well as for use in the Chapter 2 and Priority School 
District Grants. 

Students who score below the remedial standard(s) are eligible for services 
provided for in EERA legislation. Remedial standards were established by the 
State Board of Education acting on the recommendations of committees that 
represented Connecticut citizens and educators. The standard-setting 
committees recommended the following remedial standards: 

1. In mathematics, a student who answers fewer than 78 of the 144 items 
(54X) correctly is required to receive further diagnosis by the local 
school district and, if necessary, to be provided with remedial 

assi stance. 

2. In reading, a student whose Degrees of Reading Power (DRP) unit score 
is lower than 55 is required to receive further diagnosis and, if 
necessary, to be provided with remedial assistance. 

3. In writing, a student receiving a total holistic score less than 4 is 
required to receive further diagnosis by the local school district 
and, if necessary, to be provided with remedial assistance. 



The mastery and remedial standards were established by the State Board of 
Education on June 4, 1986. For a detailed explanation of the remedial 
standard-setting process, see Appendix D (p. 43). 



STATEWIDE ACHIEVEMENT GOALS 

In addition to mastery and remedial standards, statewide achievement goals 
have been established in the content areas of mathematics, reading (DRP) and 
writing. These goals represent high expectations and high levels of 
achievement for Connecticut public school students. 

The achievement goals are as follows: 

0 In mathematics, all students must master 31 of 36 objectives tested. 

0 In reading, a student must score a Degree of Reading Power (DRP) unit 
score of 62 with Q0% comprehension. 

0 In writing, a student must score a total holistic score of 7 on a 
scale of 2 to 8. 
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STUDENT GROWTH OVER TIME 



The Connecticut Mastery Test (CMT) Program is designed to provide 
criterion-referenced information about the level of student mastery of 
objectives in grades four, six and eight. However, the basic scores reported 
for the mastery tests do not provide a system for evaluating achievement 
growth from grade four to grade six to grade eight. This is so because 
mastery decisions are based on student performance (mastery/non-mastery) on 
objectives that are unique to grade level. Mastery of objectives cannot be 
compared directly across grade levels and tests because of the differences in 
the number of objectives, curriculum content and levels of difficulty. In 
order to make valid interpretations across grade levels, the mastery test 
performance must first be linked using a procedure called vertical equating. 

Purpose of Vertical Equating 

Vertical equating is a psychometric technique for comparing tests at all 
ability levels. This is accomplished by putting them on a new scale which is 
common to the tests. Vertical equating is based on two assumptions. The 
first is that learning is continuous. The second is that instruction in each 
area is related to increased achievement in that area. These assumptions 
enable test developers to create a scale score that covers a wide range of 
content over several grades. The development of these "growth scales" is a 
common practice and has been used successfully in the development ov a variety 
of achievement test batteries. The purpose of vertical equating is to provide 
one scale score system which can be used to compare performance across 
multiple grade levels. This score system enables test users to interpret test 
score information over time without altering the basic nature of the testing 
program. This achievement growth can be monitored over time on the basis of 
student performance on the CMT across grades. 

Development of Vertical Scales 

In order to develop a vertical scale, performance on the grade four, grade six 
and grade eight mastery tests was statistically linked. This was accomplished 
during the 1987 administration of the CMT using representative statewide 
samples of approximately 5,000 sixth-grade students and approximately 7,000 
eighth-grade students. Each group of students at grade six and grade eight 
was administered the appropriate on-grade level test form of the CMT along 
with one below-grade level section of the CMT. Specifically, each group of 
eighth-grade students took the grade eight test as usual and a part of the 
grade six test. Likewise, each sixth-grade group took the grade six test as 
usual along with a section of the grade four test. Each sample of students 
took only one below-level section of the CMT involving approximately one hour 
of additional testing time. Performance on the below-level items was not 
counted toward the CMT scores of Individual students. For each of these 
linking samples, item difficulty estimates were obtained for the on-grade and 
below-grade level items by analyzing all items together as one test. Once 
items from the on-grade and below-grade level tests were linked, item 
difficulties from each level of the CMT were adjusted to a common metric to 
produce the vertical scale. 
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Vertical scales were established in the content areas of mathematics and the 
reading comprehension section of the language arts test. For each grade and 
content area, every correct score corresponds to a specific value on a common 
score scale (vertical scale). Each of the vertical scales was constructed so 
that each scale score point represents the same theoretical achievement level 
whether derived from a score on the grade four test, a score on the grade six 
test, or a score on the grade eight test. This allows valid interpretations 
of growth across time using tests differing in content, length and item 
difficulty. All items on the mathematics and reading comprehension tests were 
used in the development of the vertical scales. The writing and language arts 
tests were not scaled because of the nature of these assessment processes. 
The Degrees of Reading Power (DRP) test employs DRP unit scores which are 
already on a common scale across grades, obviating the need for any other 
development. (For more information see Congero, W.J., 1989, The Development 
of Vertical Scales to Enhance the Evaluation of Assessment Data. Paper 
presented at the annual conference of the National Council of Measurement in 
Education, San Francisco, OK. This paper is available through the Student 
Assessment and Testing Unit of the Bureau of Evaluation and Student 
Assessment. ) 

Scaled scores can be used to measure growth over time because CMT scores from 
all three grade levels have been placed on a common scale. These scales 
provide a means of monitoring students' academic progress from grade to 
grade. Before the scales were developed, it was difficult to assess the 
performance of groups of test takers as they moved from grade to grade because 
of differences in test length, curriculum content covered and levels of 
difficulty on the fourth-, sixth- and eighth-grade tests. 

Since students who took the fourth-grade test in 1988 subsequently took the 
sixth-grade test in 1990, change in test performance can be assessed across 
two years' time. Similarly, change In performance can be assessed for 1991 
sixth graders who took the grade four test in 1989. A summary of the overall 
growth in performance for these two groups of students in the content areas of 
mathematics and reading comprehension can be found in the 1991-92 Grade 6 
Summary and Interpretations Manual, Students who took the fourth-grade tests 
in 1986 subsequently took the sixth-grade test in 1988 and the eighth-^grade 
test in 1990. Similarly, students who took the fourth-grade test in 1987 
subsequently took the sixth-grade test in 1989 and the eighth-grade test in 
1991. A summary of the overall growth in performance for these groups of 
students in the content areas of mathematics and reading comprehension can be 
found on pages 30 and 31 of this manual. 



NORMATIVE INFORMATION 

The CMT Program is designed to provide detailed information about fourth--, 
sixth- and eighth-grade students* mastery of specific skills and objectives. 
The provision of national norms with CMT results is intended to enhance the 
usefulness and flexibility of mastery test information by offering a bridge to 
conventional norm-referenced testing programs. The decision to provide 
normative information with the CMT does not change the essential purposes of 
our criterion-referenced testing program. The CMT will continue to be used 
for diagnostic and other instructional purposes with results reported at the 
student, classroom, school, district and state levels. 
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In particular, national norms provide greater: 

0 Test rconomy. By providing national norms with CMT results, school 
districts can eliminate their standardized testing programs at these 
grades, thus saving money and undue testing time while retaining 
normative data, 

0 Test Efficiency. Federal compensatory programs require the 

systematic testing of students using instruments that can provide 
normative information. Because norms are provided with the CMT, 
school districts will not have to "double test" compensatory program 
students. This service allows for increased instructional time for 
these students, 

0 Test Interpretability. Criterion-referenced test (CRT) programs 

may be criticized because the public has difficulty interpreting CRT 
performance. National norms will assist in the interpretation of CMT 
performance by providing a traditional benchmark with which the 
public is familiar. 

Development of Norms 

In order to provide estimated national norm-referenced data based on CMT 
performance, items on the CMT were statistically linked to items on a 
nationally norm-referenced test (NRT), Content-appropriate items from a 
nationally normed host test were included on the CMT to provide a common 
referent to both tests. Test equating procedures were then used to link CMT 
items with the normed test by placing all the items on a common scale. With 
this linkage in place, estimates of how the performance of Connecticut 
students compares to a national sample could be made. The NRT used to 
accomplish this task was the sixth edition of the Metropolitan Achievement 
Test (MAT-6), normed in 1986, The equating of the CMT to the MAT-6 enabled 
group summary scores on the CMT to be interpreted relative to the MAT-6 
nationally representative normative data. 

The CMT was initially equated to the MAT-6 during the pilot testing phase to 
investigate the relationship of the test content and material between the two 
tests and the differential nature of the items included on the CMT and MAT-6, 
In addition, these preliminary data provided a benchmark by which the 
stability of the link could be monitored over time. The stability issue is 
monitored each year by readministering MAT-6 items during CMT administrations 
using representative statewide samples. The comparison of these data with 
prior information provides the information necessary to identify the 
instructional effects on student performance over time and to update the 
CMT/MAT-6 link as appropriate. This monitoring and updating ensures the 
continued accuracy of the normative estimates. 



RESEARCH OPTIONS PROGRAM 

The Research Options Program is a free service provided by the Connecticut 
State Department of Education (CSDE) to help educators and educational 
policymakers gain access to the extensive information available from the 
Connecticut Mastery Test (CMT), Participation in the Research Options Program 
is completely voluntary. 



The Research Options Program allows educators and educational policymakers 
(i.e., superintendents, principals, researchers, evaluators and school board 
members) to benefit from customized research investigations designed to suit 
their individual needs or questions. Many school districts have taken 
advantage of the Research Options Program in previous years to successfully 
address special local concerns. 

The Research Options Program provides a number of ways of examining student 
achievement, as measured by the CMT, For example, one method is to compare 
aggregated student test scores obtained from the CMT in two or more categories 
of interest. Categories might include males and females, special program 
students compared to non-special program students, or any other comparison. 
These reports include tables that show the proportion of students mastering 
each objective, average number of objectives mastered and the achievement 
indicators for students on each component of the test under consideration. 
These breakdowns allow district personnel to directly compare the performance 
of specific groups of students. In addition, graphics are provided, as 
appropriate, with each report in order to simplify the task of interpreting 
data. 

The Research Options component of the CMT has grown a great deal since the 
first study was performed on the Connecticut Basic Skills Proficiency Test 
almost a decade ago. This year, test directors and evaluators in 26 districts 
took advantage of this valuable resource to address questions of local 
interest. In addition, statewide programs such as Bilingual Evaluation, 
Chapter I and School Effectiveness have used the research options to obtain 
useful information for participants in over 100 districts, [For more 
information see Mooney, R,F,, 1989, The Connecticut Mastery Test Research 
Options Program: The Application of State Criterion-Referenced Test Reports 
for Local Research Needs, Paper presented at the annual conference of the 
National Council of Measurement in Education, San Francisco, CA, See also the 
Research Options Handbook (1988) provided by the Connecticut State Department 
of Education, (These references are available through the Student Assessment 
and Testing Unit of the Bureau of Evaluation and Assessment,)] 



TEST ADMINISTRATION AND SCORING 

The regular administration of the Connecticut Mastery Test (CMT) for 1991 was 
conducted using Form D during a three-week period commencing on September 23, 
1991. Test sessions were conducted by local school district staff under the 
supervision of local test coordinators who had been trained by staff of the 
Connecticut State Department of Education (CSDE) and The Psychological 
Corporation (TPC), A student who took all subtests participated in 
approximately eight hours of testing. 

The Grade 8 Connecticut Mastery Test had eight testing sessions. 

Mathematics I (60 minutes) 

Mathematics II (60 minutes) 

Mathematics III (60 minutes) 

Writing Sample (45 minutes) 

Degrees of Reading Power (70 minutes) 

Reading Comprehension (60 minutes) 

Listening Comprehension (45 minutes) 

Writing Mechanics/Study Skills (60 minutes) 
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At the conclusion of the make-up testing period, answer booklets were returned 
to TPC in San Antonio, Texas for optical scanning and scoring, and then 
organized in preparation for holistic scoring workshops. 

Scoring of the Language Arts and Mathematics Tests 

The mathematics and language arts multiple-choice tests were machine-scored by 
TPC- Mathematics scores were reported for the total test as well as for 
mastery by each objective* Language arts scores were reported for mastery of 
each objective only. 



Scoring of the Writing Sample 

Every writing sample was scored by Connecticut educators using a technique 
known as the holistic scoring method. Holistic scoring is an impressionistic 
and quick scoring process that rates written products on the basis of their 
overall quality. It relies upon the scorers' trained understanding of the 
general features that determine distinct levels of achievement on a scale 
appropriate to the group of writing pieces being evaluated. All participants 
received on-site training and were required to demonstrate a clear 
understanding of the scoring criteria prior to actually scoring student 
essays. Each paper receives a final score between 2 and 8, where 2 represents 
a poor paper and 8 represents a superior paper. A thorough description of the 
training and scoring process, including sample papers representing different 
holistic scores, is presented in Appendix E (p. 49). 

In past years, the Connecticut Mastery Test writing sample was scored 
exclusively by Connecticut teachers. Following this year's scoring sessions, 
it was discovered that some of the Connecticut papers had not been scored 
accurately according to the established holistic scoring criteria. This 
problem was identified by department staff while conducting routine 
post-scoring checks. Further examination of the scoring process by department 
staff and contractors established that a portion of the papers in grades 6 and 
8 would need to be rescored. Accordingly, all grade 6 and grade 8 papers were 
submitted to a validation rescoring by our scoring contractor. This effort 
succeeded in confirming the accuracy of the Initial holistic scores for the 
large majority of papers. Where a discrepancy was detected, the original 
scores were corrected to be made consistent with the established criteria. 



Analytic Scoring 

All papers receiving holistic scores at or below the remedial standard of 4 
also received analytic scoring in four categories (traits): focus, 
organization, support/elaboration and conventions. Analytic scoring is a 
thorough, trai t-by-trai t analysis of those components of a writing sample that 
are considered important to any piece of writing in any context. This scoring 
procedure can provide a comprehensive picture of a student's writing 
performance if enough traits are analyzed. It can identify those traits that 
make a piece of writing effective or ineffective. However, the traits need to 
be explicit and well defined so that the raters understand and agree upon the 
basis for making judgments about the writing sample. The analytic rating 
guide and sample marker papers for the analytic scoring are presented in 
Appendix F (p. 61). 




Scoring of the Degrees of Reading Power (DRP) Test 

The DRP multiple-choice test was machine-scored by TPC. The scores reported 
are in DRP units. These scores identify the difficulty or readability level 
of prose that a student can comprehend. This makes it possible to match the 
difficulty of written materials with student ability. These scores can be 
better interpreted by referring to the readability levels of some general 
reading materials as shown below: 

0 Elementary textbooks (grades 7 through 9) - 5^-65 DRP Units 

0 Personality Section - teen magazines - 55 DRP Units 

0 Adult General Interest Magazines - fiction - 60 DRP Units 

A much more extensive list of reading materials is contained and rated in the 
Readability Report , Seventh Edition, published by The College Board. 

The conversion between DRP unit scores and raw scores can be made from the 
tabled values obtainable through the Student Assessment and Testing Unit of 
the Bureau of Evaluation and Student Assessment. 



SCHOOL DISTRICT TEST RESULTS REPORTING 

The CMT school district reports are designed to provide useful and comprehensive 
test achievement information about districts, schools and students. Four standard 
test reports are generated to assist superintendents, principals, teachers, 
parents and students to understand and use criterion-referenced test results. 
Appendix G (p. 67) presents samples of the district, school, class and 
parent/student diagnostic score reports. 



FALL 1991 STATEWIDE TEST RESULTS 

The Grade 8 Connecticut Mastery Test provides a comprehensive evaluation of 
student performance on specific skills that Connecticut educators feel are 
important at the beginning of eighth grade. The mastery test's greatest 
instructional utility lies in its identification of areas of student weakness and 
strength. This report profiles the statewide results. Each school district also 
receives a full complement of reports that identify patterns of academic strength 
and weakness at the district, school, classroom and individual student levels. 

Chart 1 (p. 12) gives a statewide summary of the average number of objectives 
mastered (mathematics and language arts), average writing and reading scores, the 
number of students scored, the number of students scoring at or above the remedial 
standard and goal (where applicable) and the percent of students scoring at or 
above the remedial standard and goal (where applicable). 
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The following are highlights of the 1991 Grade 8 CMT results: 
MATHEMATICS 

0 Eighth graders mastered an average of 25,8 of the 36 objectives tested, 
up slightly from last year*s figure of 25-7, 

0 A total of 87,8% of the students scored at or above the remedial 
standard, equaling last year's figure of 87. Qt. 

0 A total of 38, 1% of the students scored at or above the mathematics 
goal, an increase from last year's figure of 37%, 

LANGUAGE ARTS 

0 Eighth graders mastered an average of 8-3 of the 11 objectives tested, 
down slightly from last year's figure of 8,4, 

WRITING 

0 Eighth graders averaged 5,6 on a scale of 2 to 8, up slightly from last 
year's 5.5. 

0 A total of 92.51 of the students scored at or above the remedial 
standard, down slightly from last year's figure of 93. 2X. 

0 A total of 28.4% of the students scored at or above the writing goal, an 
increase from last year's figure of 27%. 

READING 

0 Eighth graders averaged 63 units on the Degrees of Reading Power (DRP) 
test, equaling last year's figure. 

o A total of 80.5% of the students scored at or above the remedial 
standard, up slightly from last year's figure of 79.7%. 

0 A total of 63.9% of the students scored at or above the reading goal, up 
from last year's figure of 62%. 
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CHART 1 

1991 CONNECTICUT MASTERY TEST RESULTS 
GRADE 8 STATEWIDE SUMMARY 



AVERAGE 

NUMBER OF NUMBER OF STUDENTS AT OR ABOVE STUDENTS AT OR ABOVE 

OBJECTIVES STUDENTS REMEDIAL STANDARD* STATE GOAL** 

SUBJECT MASTERED SCORED NUMBER PERCENT NUMBER PERCENT 

MATHEMATICS 25.8 30,586 26,864 87.8% 11,654 38.1% 

LANGUAGE ARTS 8.3 30,493 



AVERAGE 
HOLISTIC SCORE 

WRITING SAMPLE 5.6 30,664 28,372 92.5% 8,717 28.4% 



AVERAGE DRP 
UNIT SCORE 

READING 63 31,060 25,015 80.5% 19,857 63.9% 



* MATHEMATICS REMEDIAL STANDARD = 78 ITEMS CORRECT 
WRITING REMEDIAL STANDARD = 4 
READING REMEDIAL STANDARD = 55 DRP UNITS 



" MATHEMATICS GOAL = 31 OBJECTIVES MASTERED 
WRITING GOAL = 7 

READING GOAL = 62 DRP UNITS 
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Mathematics 

In mathematics, eighth graders mastered an average of 25.8 objectives, or 
71.7%, of the 35 objectives tested. While the state's goal is that all 
students master every objective, an interim goal of 31 of the 36 objectives 
has been established which represents a high level of mathematics 
achievement. Chart 2 (p. 15) illustrates that, statewide, students 
demonstrated strength (85% or more students achieving mastery) in the 
conceptual understanding objectives of rounding whole numbers; identifying 
points on number lines, scales and grids; computational skills objectives 
involving addition, subtraction, multiplication and division of whole numbers 
and decimals; and the applications objectives of using a calculator to add, 
subtract, multiply and divide; and interpreting graphs, tables, and charts. 
However, students did not perform as effectively (fewer than 50% of the 
students achieving mastery) on the computational skills objective of 
multiplying fractions and mixed numbers; and objectives that require 
higher-level thinking — that is, solving problems involving measurement, 
measuring and determining perimeters and areas and making measurement 
conversions within systems. 

These results indicate a consistent pattern throughout the mathematics 
subtests of student strengths in primarily computational skills and one-step 
routine applications. These strengths are offset by a pattern of student 
weaknesses on higher-order objectives. For example, students are consistently 
strong in their ability to compute with whole numbers. However, there is 
consistent weakness in working with fractions and mixed numbers and solving 
problems involving measurement and measurement conversions. 

Students getting fewer than 78 questions correct on the 144-question 
mathematics section (12.2% of the students tested) were identified as needing 
further diagnosis and possible remedial instruction. 



Language Arts 

In language arts, eighth-grade students averaged 8.3 objectives, or 75.5%, of 
the 11 objectives tested. The state's goal is that all students master every 
objective. Chart 3 (p. 15) illustrates that students did reasonably well on 
writing mechanics, study skills, and literal reading comprehension. However, 
weaknesses were found in the higher order objectives of inferential and 
evaluative reading comprehension and in listening comprehension. These 
results indicate that students need to learn more effective comprehension 
strategies while simultaneously being exposed to a wide variety of reading 
selections. 



-13- 



In writing, eighth-grade students averaged 5.6 points on a scale of 2 through 
8. The state*s goal is that all students be able to produce an organized, 
well-supported piece of writing, that is, a holistic score of 7 or 8. Chart 4 
(p. 17) illustrates that 29% of the students produced an organized, 
well-supported piece of writing (scores of 7 or 8), and an additional 50% 
produced a paper which is generally well organized (scores of 5 or 6). A 
total of 14% of the students scored a 4, which indicates minimany proficient 
writing, while the remaining 7% scored below the remedial standard (scores of 
2 or 3). 

In reading (Degrees of Reading Power test), eighth-grade students averaged 63 
units on a scale of 15 through 99. The state's goal is that all students be 
able to read with high comprehension those materials typically used at the 
eighth grade or above; that 1s, at least 62 on the DRP unit scale. Chart 5 
(p. 18) illustrates that 64% of the students scored at least 62 on the DRP 
score scale, 17% scored between 55 and 61 and 19% scored below the remedial 
standard of 55. The average score of 63 suggests that the typical Connecticut 
eighth grader can read and comprehend materials normally used up to grade 
eight. To improve reading performance, more emphasis needs to be placed on 
reading nonfiction materials during the primary and intermediate grades. 
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CHART 4 
WRITING SAMPLE: 
PERCENT OF STUDENTS AT EACH SCORE POINT 
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This bar chart illustrates the distribution of students who received each 
holistic writing score, statewide. Holistic writing scores are interpreted as 
follows: a student who scores 7 or 8 has produced a paper which is well 
written with developed supportive detail; a student who scores 5 or 6 has 
produced a paper which is generally well organized with supportive detail; a 
student who scores 4 is minimally proficient; and a student who scores 2 or 3 
is in need of further diagnosis and possible remedial assistance. 



CHARTS, 

DEGREES OF READING POWER® (DRPf: 
PERCENT OF STUDENTS AT SELECTED RANGES OF 

DRP UNIT SCORES 
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This bar chart illustrates the distribution of students, statewide, scoring in each of three 
Degrees of Reading Power (DRP) score categories. DRP score categories are interpreted 
as follows; a student who scores 62 DRP units or above has met the statewide Reading 
Goal and can read, with high comprehension, materials which are typically used at grade 8 
or above; a student who scores 55-61 DRP units can read, with high comprehension, 
materials which are typically used below grade 8 but above the Remedial Standard; and a 
student who scores 54 DRP units or below is in need of further diagnosis and possible 
remedial assistance. 
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COMPARISON OF 1986 THROUGH 1991 TEST RESULTS 



Charts 6-12 (pp. 21-27) address the comparison of the 1986 through 1991 test 
results. Charts 6 (p. 21), 9 (p. 24) and 10 (p. 25) present a comparison of 
statewide average scores on the four subtests, a comparison of the percent of 
students scoring at or above the remedial standard and a comparison of the 
percent of students scoring at or above the statewide goals, respectively. The 
remaining four charts provide a comparison of the percent of students achieving 
mastery in each mathematics objective (Chart 7. p. 22) and each language arts 
objective (Chart 8. p. 23). a comparison of student achievement in relation to 
the remedial standards (Chart 11. p. 26), and a comparison of student 
achievement in relation to the statewide goals (Chart 12. p. 27). 

Chart 6 (p. 21) shows that the statewide average scores increased in all areas 
tested when 1991 results are compared to 1986 results. In mathematics, the 
average number of objectives mastered increased from 23.7 in 1986 to 25.8 in 
1991. Mathematics scores have increased slightly each year from 1986 to 1991 
indicating a positive trend. The average DRP unit score has increased two DRP 
unit points, moving from 61 in the initial assessment in 1986 to 63 in 1991. 
The average number of language arts objectives mastered has increased from 7.5 
objectives in the initial 1986 assessment to 8.3 in 1991. In writing, the 
average holistic score has risen from 5.0 in 1986 to 5.6 in 1991. 

Chart 7 (p. 22) lists the percent of students at mastery for each of the 36 
mathematics objectives. From 1986 to 1991, 29 objectives have shown a gain in 
percent of students at or above mastery, 3 have declined and 4 are unchanged. 
A comparison of the 1991 and 1986 results shows large gains (at least 10 
percentage points) in the percent of students meeting the mastery standard in 
the following objectives: ordering fractions and decimals, identifying points 
on number lines, scales and grids; identifying appropriate procedures for 
estimating fractions and decimals; identifying correct decimal point in 
multiplication/division of decimals; adding and subtracting fractions and mixed 
numbers; interpreting graphs, tables and charts; solving problems involving 
measurement; measuring and determining perimeters and areas; and making 
measurement conversions within systems. 

Chart 8 (p. 23) lists the percent of students at mastery for each of the 11 
language arts objectives. From 1986 to 1991, 10 objectives have shown a gain 
in percent of students at or above mastery, and one objective has shown a 
slight decline. When 1991 results are compared with 1986, areas which showed 
large gains (at least 10 percentage points) in the percent of students at 
mastery are: spelling, agreement and tone in writing mechanics; and literal 
and inferential reading comprehension. 

Chart 9 (p. 24) compares the percent of students who scored at or above the 
remedial standard in mathematics, writing and reading (DRP) for 1986 through 
1991. In each content area there has been a gain in the percent of students 
meeting the remedial standard from 1986 to 1991. The remedial standard for 
mathematics is 78 out of 144 items correct. There was a 5 percentage point 
increase in mathematics performance at or above the remedial standard from 1986 
(83%) to 1991 (88%). The remedial standard for writing is 4 on a scale from 2 
to 8. A 10 percentage point increase in writing performance at or above the 
remedial standard occurred from 1986 (83%) to 1991 (93%). The remedial 
standard for reading (DRP) is 55 DRP units. An 8 percentage point increase in 
performance at or above the remedial standard was reported from 1986 (73%) to 
1991 (81%). 
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Chart 10 (p. 25) compares the percent of students scoring at or above the 
statewide goals in mathematics, writing and reading from 1986 through 199U 
There has been a gain in the percent of students reaching the statewide goal 
in each of the three content areas over the six CMT administrations. In 
mathematics, the goal is 31 of 35 objectives mastered. There was an 11 
percentage point increase in performance at or above the statewide goal from 
1986 (27%) to 1991 (38%). In writing, the goal is 7 on a scale of 2 to 8. 
The percent of students scoring at or above the statewide standard increased 
from 20% in 1986 to 28% in 1991. In reading (DRP) the statewide goal is 62 
DRP units with 80% comprehension. There was a 7 percentage point increase in 
performance at or above the goal from 1986 (57%) to 1991 (64%). 

Chart 11 (p. 26) is a comparison of student achievement in relation to the 
remedial standards in 1986 through 1991. Over the six-year period, the 
percent of students at or above the remedial standard on all three tests 
(mathematics, reading, writing) has increased from 62.3% in 1986 to 72.5% in 
1991, while the percent of students below the remedial standard on all three 
tests has declined from 6.2% in 1986 to 2.3% in 1991. The percent of students 
below the remedial standard on one or more subtests has also dropped from 
36.3% in 1986 to 25.5% in 1991. 

Chart 12 (p. 27) is a comparison of student achievement in relation to the 
goals for 1986 through 1991. Over the six-year period, there has been an 
increase in the percent of students reaching the statewide goal on all three 
tests (mathematics, reading, and writing), while the percent of students below 
the statewide goal on all three tests has declined from 38.2% in 1986 to 28.0 
in 1991. The percent of students above the statewide goal on one or more 
subtests has increased from 60.0% in 1986 to 69.4% in 1991. 



Test Results by District 

Appendices H and I address the comparison of test scores by school district. 
Appendix H (p. 81) and Appendix I (p. 89) present a listing of the mathematics 
and language arts test results, respectively, for each Connecticut school 
district. In each appendix, school districts are listed alphabetically, 
followed by regional school districts. The Type of Community (TOO 
designation In the second column and the Education Reference Group (ERG) 
designation in the third column indicate the TOC and ERG groups with which 
each district or school has been classified. Definitions of the TOC and ERG 
classifications are provided in Appendix J (p. 97) and Appendix K (p. 99), 
respectively. TOC and ERG summaries follow the alphabetical listings of 
school district results in mathematics and language arts. 

The State Department of Education advises against comparing scores between and 
among school districts. It is more meaningful to compare district results 
longitudinally within each district. It is also not appropriate or meaningful 
to sum across the different tests and subtests for comparative purposes 
because of differences in test length, mastery criteria and remedial 
standards. These comparisons are inappropriate because it is impossible to 
identify, solely on the basis of this information, how the average student has 
performed in t^B districts being compared. Average scores and standard 
deviations provide more appropriate comparative information on how well the 
average student is performing, although many factors may affect the 
comparability of these statistics as well. 
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CHART 7 

MATHEMATICS: COMPARISON OF THE PERCENT OF STUDENTS 
ACHIEVING MASTERY IN EACH OBJECTIVE FOR 1986 THROUGH 1991 



UbJLw 1 IVL 


PERCENTAGE OF STUDENTS 
AT MASTERY 


PERCENTAGE 
POINT 
GAIN FROM 
1 986 TO 1 991 




1986 


1987 


1988 


1989 


1990 


1991 




CONCEPTUAL UNDERSTANDINGS 
















1 . ORDER FRACTIONS 


58% 


57% 


57% 


69% 


73% 


74% 


16% 


2. ORDER DECIMALS 


58% 


63% 


65% 


61% 


73% 


73% 


15% 


3. ROUND WHOLE NUMBERS 


87''e 


85?o 


86% 


89'*o 


92% 


92'*o 


5% 


4. ROUND DECIMALS TO NEAREST ^.^..0^ 


66''o 


71% 


72% 


67% 


74% 


73% 


7% 


5 MULT/DIV WHOLE # S & DECIMALS BY 10. 100. 1000 


67% 


75% 


75% 


73% 


75% 


73% 


6% 


6. ID FRACTIONS. DECIMALS, PERCENTS FROM PICTURES 


58% 


58% 


5Q% 


55*50 


63% 


65% 


7-0 


7. CONVERT FRACTIONS TO DECIMALS & VICE VERSA 


7A% 


70*0 


71% 


74% 


77% 


77% 


3% 


8. CONVERT FRACT/DECIMALS TO PERCENTS & VICE VERSA 


72^0 


78% 


78% 


80% 


75% 


76% 


4**o 


9. IDENTIFY POINTS ON NUMBER LINES, SCALES. GRIDS 


85'*o 


91% 


91% 


95% 


95% 


95% 


10% 


10. IDENTIFY RATIOS AND FRACTIONAL PARTS FROM DATA 


85% 


85% 


85% 


95% 


78% 


78% 


-70.C 


11. ID APPROP PROCEDURE FOR ESTIMATING FRACT/DEC 


64% 


78% 


80'''o 


70% 


76% 


77% 


13% 


COMPUTATIONAL SKILLS 
















12. ADD AND SUBTRACT WHOLE NUMBERS < 10.000 


94% 


96*'o 


96% 


95% 


96% 


95% 


1% 


13. MULT/DIVIDE 2- & 3-DlGlT # S BY 1- & 2'DlGtT #'S 


95''o 


96*' o 


96% 


96% 


96% 


95**o 


0% 


14 ADD AND SUBTRACT DECIMALS IN HORIZONTAL FORM 


89% 


85% 


85% 


88% 


87% 


87% 


-2% 


15 ID CORRECT DECIMAL POINT IN MULT/DIV OF DECIMALS 


60% 


76% 


75% 


73% 


72*o 


70% 


10% 


16. ADD/SUBTRACT FRACTIONS AND MIXED NUMBERS 


39% 


49% 


A9% 


48% 


56% 


55% 


16% 


17 MULTIPLY FRACTIONS AND MIXED NUMBERS 




51% 


49% 


47''o 


49% 


47% 


3% 


18. DETERMINE PERCENT OF A NUMBER 


49% 


54''o 


53% 


54% 


53% 


53% 


4% 


19 ESTIMATE SUMS/DIFFS OF WHOLE #'S AND DECIMALS 


78% 


77% 


78*»o 


87% 


81% 


82% 


4*0 


20. ESTIMATE PROD/QUOT OF WHOLE #'S AND DECIMALS 


69% 


70** o 


70" 0 


72% 


62% 


62% 


-7*0 


21. EST FRACTIONAL PARTS/PERCENTS OF WHOLE #'S & S 


51% 


54% 


54% 


53% 


58% 


59% 


8*-o 


PROBLEM SOLVING/APPLICATIONS 
















22 ADD/SUBT/MULT/DIV WITH A CALCULATOR 


98% 


99% 


98% 


98% 


98% 


98% 


0*0 


£0 IN 1 tnrnt 1 unArno, 1 ADLto, AINU OrlMn I o 


67% 


76% 


76% 


81% 


84% 


85% 


18% 


24. SOLVE 1- & 2-STEP PROBS-WHOLE #'S/DEC/AVERAGES 


78% 


74% 


75% 


75% 


82«o 


83*0 


5*o 


25 SOLVE 1-AND 2-STEP PROBLEMS-FRACTIONS 


54**^ 


51% 


51% 


53% 


60% 


60% 


6*b 


26. SOLVE PROBLEMS INVOLVING MEASUREMENT 


31% 


32*»o 


32*-o 


33% 


40% 


42% 


11% 


27. SOLVE PROBS INVOLVING ELEMENTARY PROBABILITY 


61*o 


62% 


63% 


60% 


65% 


66% 


5*/o 


28. ESTIMATE REASONABLE ANSWER TO A GIVEN PROBLEM 


76% 


81% 


80*''o 


83% 


80% 


81% 


5*0 


29 SOLVE PROBLEMS WITH EXTRANEOUS INFORMATION 


70*''o 


72% 


73% 


78% 


71% 


72% 


2*0 


30. IDENTIFY NEEDED INFO IN PROBLEM SITUATIONS 


76% 


79^.0 


79% 


83% 


75% 


76% 


0% 


31. SOLVE PROCESS PROBLEMS-ORGANIZ'NG DATA 


46% 


53% 


53*-o 


55% 


54% 


54% 


8*0 


MEASUREMENT/GEOMETRY 
















32 IDENTIFY FIGURES USING GEOMETRIC TERMS 


57% 


66*^0 


66% 


64% 


61% 


61% 


4% 


33. MEASURE AND DETERMINE PERIMETERS AND AREAS 


30% 


40*^0 


41% 


37% 


39% 


40% 


10*0 


34 ESTIMATE LENGTH/AREAA/OLUME/ANGLE MEASURE 


63% 


69% 


71% 


71-6 


69% 


69% 


6*6 


35. SELECT APPROPRIATE METRIC/CUSTOMARY UNIT 


78*»o 


81% 


82^o 


76^6 


78% 


78*0 


0*0 


36 MAKE MEASUREMENT CONVERSIONS WITHIN SYSTEMS 


32% 


42% 


43% 


43% 


48% 


48% 


16*o 
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CHART 8 

LANGUAGE ARTS: COMPARISON OF THE PERCENT OF STUDENTS 
ACHIEVING MASTERY IN EACH OBJECTIVE FOR 1986 THROUGH 1991 



OBJECTIVE 


PERCENT OF STUDENTS 
AT MASTERY 


PERCENTAGE 
POINT 
GAIN FROM 
1986 TO 1991 






1987 


1988 


1989 


1990 


1991 




WRITING MECHANICS 
















1 . CAPITALIZATION AND PUNCTUATION 


72% 


70% 


71% 


73% 


72% 


70% 


-2% 


2. SPELLING 


66% 


63% 


63% 


58% 


76% 


76°-o 


10% 


3. AGREEMENT 


76*o 


81% 


81% 


79% 


92% 


91% 


15% 


4. TONE 


77% 


69% 


70% 


70% 


88% 


87% 


10% 


STUDY SKILLS 
















5. LOCATING INFORMATION 


83% 


88°o 


87% 


89% 


85% 


85% 


2% 


6. NOTETAKING AND OUTLINING 


73% 


71% 


71% 


78% 


79% 


79% 


6% 


LISTENING COMPREHENSION 
















7. LITERAL 


59% 


68% 


67% 


78'^o 


63% 


62% 


3% 


8. INFERENTIAUEVALUATIVE 


62% 


66% 


66% 


67% 


66% 


65% 


3% 


READING COMPREHENSION 
















9. LITERAL 


70% 


75% 


75% 


79% 


82**'o 


82% 


12% 


10. INFERENTIAL 


54% 


57% 


58% 


58% 


66% 


66% 


12% 


11. EVALUATIVE 


57% 


61% 


62% 


66% 


64% 


64% 


7% 
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CHART 9 

COMPARISON OF THE PERCENT OF STUDENTS 
SCORING AT OR ABOVE THE REMEDIAL STANDARD 
IN EACH SUBJECT AREA FOR 1986 THROUGH 1991 
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CHART 10 

COMPARISON OF THE PERCENT OF STUDENTS 
SCORING AT OR ABOVE THE GOAL 
IN EACH SUBJECT AREA FOR 1986 THROUGH 1991 
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Normative Results 



Normative information is provided to indicate how well the average student in 
Connecticut performs compared to a national sample of students. Norms have 
been available for the mathematics test, the language arts test and the 
reading comprehension test since 1987. This year, for the second year, 
normative information is also being provided for mathematics problem solving. 
These norms are based on links established between the CMT and the sixth 
edition of the Metropolitan Achievement Test (MAT-6). The norms are expressed 
in percentile ranks which provide estimates of group performance relative to 
the performance of the national MAT-6 norm group. Percentile ranks range from 
1 to 99. A percentile rank of 50 represents the score that divides the norm 
group into two equal parts; half scoring below and half scoring above this 
value. Each reported percentile rank represents the performance of a 
nationally representative sample of students in relation to Connecticut 
student performance* 

The following are the estimated norms for the grade eight statewide averages. 
In the content areas of total mathematics, language arts and reading 
comprehension (not DRP), data are provided for the 1987 through 1991 
administrations. Normative information in the content area of mathematics 
problem solving is presented for the 1990 and 1991 administrations only. 

Grade Eight 



Total Mathematics 
Language Arts 
Reading Comprehension 
Mathematics Problem Solving 



1987 


1988 


1989 


1990 


1991 


67 


67 


67 


68 


68 


67 


69 


69 


69 


65 


57 


57 


59 


61 


59 








64 


68 



Patterns in the data are summarized below. 

0 In each content area, the mean national percentile rankings of 

Connecticut students substantially exceed the national average (50th 
percentile rank). 

0 The norms for mathematics and language arts have remained similar 
over the five years with percentile ranks ranging from 65 to 69 in 
value, Reading comprehension performance is lower than either 
mathematics or language arts when compared to a national sample, with 
percentile ranks ranging from 57 to 61 over the five administrations. 

0 The percentile ranks within each content area are quite stable across 
the five years, differing in value by no more than four points. 

It should be pointed out that these norms provide a way to interpret the 
performance of the average Connecticut student relative to a national sample. 
They do not address the issue of how Connecticut, as a state, compares to 
other states. The fact that, in 1991, the average Connecticut student is at 
the 68th percentile in mathematics does not mean that the state as a whole 
would be in the 68th percentile if it were compared to other states. A 
state-by-state achievement testing program has been endorsed by the Council of 
Chief State School Officers (CCSSO) and the National Governors' Association 
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(NGA) and is in progress using the National Assessment of Educational Progress 
(NAEP) Program. Connecticut participated in the 1990 trial state assessment 
for mathematics at grade eight. Results of this assessment were released June 
6, 1991 at a national press conference in Washington, D.C. In addition, 
Connecticut participated in the 1992 trial state assessment in grades four and 
eight. 



Norms Available to Districts 

Total mathematics, language arts, reading comprehension, and mathematics 
problem solving norms can also be provided for groups of students at the 
district level. Each year all districts are notified by the CMT contractor 
that norms for their own districts and/or schools within their districts are 
optionally available. In addition, districts are offered all materials and 
directions to hand-calculate norms for groups of students within their 
districts (e.g., Chapter I students). There is no charge for either of these 
services. Any district that requests this information receives it directly 
from the CMT contractor. No district receives normative information unless it 
is specifically requested by the superintendent. Approximately one half of 
Connecticut school districts have requested norms in the past. 



Longitudinal Results 

In order to interpret student performance across grade levels, vertical scales 
were developed in the areas of mathematics and reading comprehension. Scaled 
scores can be used to measure growth over time because CMT scores from all 
three grade levels have been placed on a common scale. These scales provide a 
means of monitoring students' academic progress from grade to grade. Before 
the scales were developed, it was difficult to assess the performance of 
groups of test takers as they moved from grade to grade because of differences 
in test length, curriculum content covered and levels of difficulty on the 
fourth-, sixth- and eighth-grade tests. 

Since students who took the fourth-grade test in 1986 subsequently took the 
sixth-grade test in 1988 and the eighth-grade test in 1990, change in 
performance on the test can be assessed across four years' time for the 
group. Similarly, change in performance can be assessed for 1991 eighth 
graders who took the grade four test in 1987 and the grade six test in 1989. 
Chart 13 (p. 30) and Chart 14 (p. 31) present overall growth in performance for 
these students in the content areas of mathematics and reading comprehension, 
respectively* These results show meaningful growth in both mathematics and 
reading comprehension for the groups of students from grade four to grade six 
and from grade six to grade eight. Chart 13, for example, shows that the 
average statewide performance in mathematics, for the group of students wh6 
took the fourth-grade test in 1986, the sixth-grade test in 1988 and the 
eighth-grade test in 1990, has moved in a positive direction. While Initial 
results are encouraging, it is premature to draw definitive conclusions about 
how much growth to expect as students progress from grade to grade. Such 
conclusions are possible only after the program has been in effect for several 
years. It should be noted that each eighth-grade group differs, to some 
extent, from its respective sixth-grade group and that each sixth-grade group 
differs from its respective fourth-grade group because some students entered, 
while other students exited the Connecticut public school system over the 
years. 



-29- 



CO 

< 
X 

o 



eo 

UJ 
Q 
< 

o 

(D 

Ul 
Q 
< 

O 

O 
H 

UJ 
O 
< 
GC 
O 

cn 
o 

UJ 
X 

5 



-o 

c ^ 

O 03 



CO 

E 

< 

CD O 
O O) 

E 



E 

< 



00 
CD 

CO 



(DO 



o 

H: 

CD 
£L 

CO 

.9 o 

CD .2 
CO 

!^ w 

:^ E 

(D < 
(DC- 

> ^ 
< 2 

o ^ 
w c 
^ o 

0-2 

CI ■*-> 

o w 
Oc 

E 

< 

CO 
CO 

o 



00 
O 

o 



c 
o 

to 2 

E o 
~ o 

EC/) 

5 CD 

CO 'co 
CD=) 

O 

o 



o 

CO 

E 

< 

CO 



^-8 



C 
CO 



O 

•C 
o 
sz 
o 
O 



CO 
CO 

i5 
O 

■c 
o 

o 
cdO 

CD 2 

cx)0 
o ^ 

CO 
■♦-^ 

CO 
CD 

DC 



o 
o 

00 




CO 5 F 



o 
o 



8JO0S 9|B0S 



ERLC 



-30- 



< 
X 

o 



00 

UJ 
Q 
< 

o 
o 

<o 

HI 
Q 
< 

o 
o 

LU 
O 
< 

cc 
o 



g 

CO 

z 

LU 
X 
LU 
OC 
Q. 

o 
o 

o 
z 

o 
< 

111 
oc 



ERIC 



■o 

c ^ 

O 03 

c E 



E 
c 

CO 



CD 



CD O) 

C - 

^ -o 



00 



coo 

:o o 

.0 

CD 



E 

CD 
CL 

o 
c 

x> 

CO 

(D CO.b o 

:2 c £0) 

CO en 

CX) T- O) 

cO o 

CD C = 
Q-O CO 

Oto.E 
■£ E 

^ CD 

O 



C 
CO 

CO 

CO 



0 
Q 



"5$ 



C CO 

o > 



o 

t: 
o 
sz 
o 
O 

CD 

■o 

CO 



CO 
CO 

i5 
O 

■c: 
o 

o 
O 



i -on 

J to = 

1 c/) 2? 

i sf - 



o = 

^ JQ 

CO to 



«/) 

CO . 
0) </) 

^ E 
oS 

to <o 

CU fJi 

£ <w 
_ to 

2R 
^3 

— OL 
O • 

oJ 
T3 <D 

QJ ^ 

<D O 

I- 

o c 
o I 

CO .2. 
^lo 

CO o 

o -ts 



CD 
CD 2 

o ^ 

CO 

CD 
DC 





1 




C ; 






Level 
of 

derstandi 


Level IV 
652+ 


Level III 
610-651 




1 




5 







CO 
0) 

<0 Or 

o5?[ 



I 0) S? c/) 
IcO O 

CO] CO CO u 

, 1/ 



^ CO 

"D CO 
0) CO 

1^ 

CO — 
E 



> CJ 



a)co 

.g £ 



- £ 



cn 

li 



3-^ 

Q> CO 
cn 



o £ c 
(/5 ^ CO 



0) «^ 



to 



5 



2o 



ILL, 



: 0) £ U, 

0IS8I 
CVJ'CO CO u 



c->'u 3 

C/) CO ^ 



■D O 

CO a 

5^ 



O [ 



coco 



CO 

O 



(0 



CO 
CD 



^11 

< 



CD 

O) ^ CO 
CO $ p 

> <St 

*^ W CD 

a 



o 
o 
00 



o 
10 



10 

CO 



in Q)S<D CO 
CO nj o c 



S 



CM 

in 



o 
o 
in 



9JO0S 9|B0S 



-31- 



Participation Rate Results 



Appendix L (p, 103) presents the number of eighth-grade students in each 
district and the percents of students who participated in the grade eight 
mastery testing during the fall 1991 statewide administration. Appendix L 
also shows the percent of students exempted from CMT testing* The 
alphabetical listing of districts provides the following information for each 
district: 



Column 1 The name of the district 

Column 2 The total eighth-grade population at the start of mastery 
testing 

Column 3 The number of students eligible for testing 
Column 4 The percent of total population exempted from testing 
Columns 5-8 The percent of eligible students tested in each content 
area 



The results in Appendix L illustrate that participation rates by school 
district on the eighth-grade CMT were quite high, with only a few exceptions. 
However, the high percentage of students exempted from the CMT, statewide, 
combined with the large variation in exemption rates among districts, has 
raised concerns about the fair application of exemption procedures and its 
impact on students. The Department has examined the impact of the exclusion 
provisions on the CMT programs for Special Education and bilingual students. 
The results from these analyses are available from the Division of Research, 
Evaluation, and Assessment. 
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Test Construction 
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I 



Test Construction 



The development of the eighth-grade criterion-referenced mastery test required 
the formation of seven statewide advisory committees. These included the 
Mathematics and Language Arts Advisory Committees, the Psychometri cs Advisory 
Committee, the Bias Advisory Committee, the Connecticut Student Assessment 
Advisory Committee (formerly the Mastery Test Implementation Advisory 
Committee), and two standard-setting committees, one for mathematics and one 
for language arts. These committees were comprised of representatives from 
throughout the state* Members were selected for their area of expertise. 
Approximately 150 Connecticut educators participated on the mastery test 
committees which met over 80 times during the first 18 months of test 
development. (See Acknowledgements, p. v and p. 48.) 

Beginning in the spring of 1985, content committees in both language arts and 
mathematics participated in each stage of the test development process, 
including assisting the State Department of Education in the selection of The 
Psychological Corporation as its test contractor. First, the content 
committees reviewed the curriculum materials prevalent throughout the state 
and the scope of the national tests in use in Connecticut at the respective 
grade levels. Additional resources included the Connecticut curriculum guides 
in mathematics and language arts, de/eloped in 1981, as well as the results of 
recent Connecticut Assessment of Educational Progress (CAEP) assessments in 
mathematics and language arts. Next, the committees identified sets of 
preliminary mathematics and language arts objectives which reflected existing 
curriculum materials and the goals of the mastery testing program. The 
content committees defined an objective as an operational ized learning outcome 
that was fairly narrow and clearly defined. 

Four criteria were used in identifying the appropriate learning outcomes or 
test objectives and in selecting specific test items to be included on the 
Grade 8 Connecticut Mastery Test (CMT). To have been considered for use, test 
objectives and items must have been: 

(1) significant and important; 

(2) developmental ly appropriate; 

(3) reasonable for most students to achieve; and 

(4) generally representative of what is taught in Connecticut schools. 

Once the objectives were identified, item specifications and/or sample items 
were written. Item specifications are written descriptions of the types and 
forms of test items that assess an objective- They also prescribe the types 
of answer choices that can be used with each item. 

After the test specifications were written and agreed upon, the test 
contractor wrote items and response choices for each of the objectives. The 
items were then reviewed by the content committees. Items which met the 
criteria of the test specifications and received the approval of the content 
committees were considered for the pilot test. Before testing, the Bias 
Advisory Committee reviewed each item for potential discrimination related to 
gender, race, or ethnicity in the language or format of the question or 
response choices. After their review was completed, the pilot test forms were 
constructed. Over 1,600 customized Connecticut items were included in the 
October 1985 grade eight pilot test in language arts and mathematics. 



The Psychometrics Advisory Committee provided advice concerning other aspects 
of the pilot test including the sampling design, statistical bias analysis, 
the design of item specifications and pilot test administration procedures. 
The recommendations proposed by the Psychometrics Advisory Committee were 
reviewed and endorsed by the Connecticut Student Assessment Advisory Committee. 



Pilot Tests 

After the items had been reviewed, twelve test forms (six in mathematics, and 
six in language arts) were piloted for the grade eight test. The purpose of 
several pilot test forms was to ensure that enough test items were included to 
construct three comparable test forms from the pilot test results. 

Over 8,000 grade eight students participated in the October 1985 pilot test. 
In January 1986, the pilot test results were made available to Connecticut 
State Department of Education (CSDE) staff. The process of selecting items to 
construct three comparable test forms began by the Bias Advisory Committee 
examining the pilot test statistics of each item for potential bias. As a 
result, some items were eliminated from the item pool. From the remaining 
items, test forms were constructed to be equivalent in content and difficulty 
at both the objective and total test levels. 

Once the items were sorted on this basis, the test contractor prepared three 
complete forms of the mathematics test and two complete forms of the language 
arts test. These forms were approved by the content committees. Each form 
was created to be equal in difficulty and test length. A third language arts 
test was constructed after a few additional items were piloted as part of a 
later test administration. Later, during subsequent CMT administrations, 
enough items were pilot tested to yield two additional test forms. The 
psychometric procedures used to construct each of these test forms focused 
primarily on the use of the one-parameter item response model. 



Survey 

In October 1985, a survey of preliminary grade eight mastery test objectives 
was sent to over 4,000 Connecticut educators. The purpose of the survey was 
to determine (1) the importance of the proposed mathematics and 
reading/language arts objectives, and (2) whether the objectives were taught 
prior to the beginning of grade eight. Approximately a 45% response rate was 
achieved which included approximately one-third of the respondents 
representing urban school districts. Thirty-six out of the original 
thirty-seven mathematics objectives were judged to be important learning 
skills. 
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APPENDIX B 
Grade Eight Mathematics Objectives 




Grade Eight Mathematics Objectives 



The 36 objectives of the eighth-grade mathematics test are listed below. 
There are four test items for each objective. The number of items in each 
domain is indicated in the parentheses. 

CONCEPTUAL UNDERSTANDINGS (44) 

1 . Order fractions 

2. Order decimals 

3. Round whole numbers 

4. Round decimals to the nearest whole number, tenth and hundredth 

5. Multiply and divide whole numbers and decimals by 10, 100 and 
1,000 

6. Identify fractions, decimals and percents from pictorial 
representations 

7. Convert fractions to decimals and vice versa 

8. Convert fractions and decimals to percents and vice versa 

9. Identify points on number lines, scales and grids 

10. Identify ratios and fractional parts from given data 

11. Identify an appropriate procedure for making estimates with 
decimals and fractions 

COMPUTATIONAL SKILLS (40) 

12. Add and subtract whole numbers less than 10,000 

13. Multiply and divide 2- and 3-digit whole numbers by 1- and 
2-digit numbers 

14. Add and subtract decimals (to hundredths) in horizontal form 

15. Identify the correct placement of the decimal point in 
multiplication and division of decimals 

16. Add and subtract fractions and mixed numbers 

17. Multiply fractions and mixed numbers 

18. Determine the percent of a number 

19. Estimate sums and differences of whole numbers and decimals 
including making change 

20. Estimate products and quotients of whole numbers and decimals 

21. Estimate fractional parts and percents of whole numbers and 
money amounts 

PROBLEM SOLVING/APPLICATIONS (with calculators available) (40) 

22. Compute sums, differences, products and quotients using a 
calculator 

23. Interpret graphs, tables and charts 

24. Solve 1- and 2-step problems involving whole numbers and 
decimals, including averaging 

25. Solve 1- and 2--step problems involving fractions 

26. Solve problems involving measurement 

27. Solve problems involving elementary probability 

28. Estimate a reasonable answer to a given problem 

29. Solve problems with extraneous information 

30. Identify needed information in problem situations 

31. .Solve process problems involving the organization of data 
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MEASUREMENT/GEOMETRY (20) 



32, Identify figures using geometric terms 

33, Measure and determine perimeters and areas 

34, Estimate lengths, areas, volumes and angle measures 

35, Select appropriate metric or customary units and measures 

36, Make measurement conversions within systems 



Performa xe on all 36 math objectives is reported at the stuuent. classroom, 
school, district and state levels. 
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Grade Eight Language Arts Objectives 



There are eleven language arts objectives and two holistic measures, one for 
reading and one for writing, within the eighth-grade language arts test. The 
number of items for each content area or objective is indicated in the 
parentheses. 



WRITING MECHANICS (39) 

1. Capitalization and Punctuation (12) 

2. Spelling (8) 

3. Agreement (15) 

4. Tone (4) 

STUDY SKILLS (16) 

5. Locating Information (12) 

6. Note-taking and Outlining (4) 

LISTENING COMPREHENSION (20) 

7. Literal (4) 

8. Inferential and Evaluative (16) 
READING COMPREHENSION (36) 

9. Literal (8) 

10. Inferential (14) 

11. Evaluative (14) 

DEGREES OF READING POWER (77) 

WRITING SAMPLE (1) 

Holistic scoring is provided for all students. Analytic scoring is 
provided for students who score at or below the remedial standard of 4 
(on a scale of 2-8). 



Performance on all eleven Language Arts objectives, the Degrees of Reading 
Power and the Writing Sample is reported at the student, classroom, school, 
district and state levels. 
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Remedial (Grant) Standard-Setting Process 



Background 

There are several acceptable strategies for setting standards on 
criterion-referenced tests- Each of the proposed methods has one or more 
unique characteristics. One common element to the various methods is that 
they all offer to the individuals who are setting the standards some process 
which reduces the arbitrariness of the resulting standard. Different methods 
accomplish this in different ways. All methods systematize the standard- 
setting process so that the result accurately reflects the collective informed 
judgment of those setting the standard. 



Types of Standard-Setting Methods 

Standard-setting methods can generally be categorized into three types: test 
question review, individual performance review and group performance review. 
Test question review methods specify a procedure for standard setters to 
examine each test question and make a judgment about that question. For 
example, standard setters might be asked to rate the difficulty or the 
importance of each question. These judgments are then combined mathematically 
to produce a standard. Individual performance review methods also require 
standard setters to make judgments, but the judgments are made on the basis of 
examining data that indicate how well individual students perform on test 
items. These data may be based on actual pilot test results or projected 
results using mathematical theories. In this method, additional student 
information, such as grades, may also be used to inform the standard setters. 
Group performance review methods provide for judgments to be made based on the 
performance of a reference group of students. That is. standard setters 
review the group performance and make a determination where the standard 
should be set based on the group results. 



Selection of a Standards-Setting Method 

Several factors af f ect ^the choice of a particular standard-setting method. 
The type of test is one consideration. For example, some methods are only 
appropriate for multiple-choice questions or for single correct answer 
questions while other methods are more flexible. For example, time 
constraints are a consideration if student performance data are necessary. 
In this case, a pilot test must be conducted and the test results must be 
analyzed prior to setting the standards. Another consideration is the 
relative importance of the decisions that will be made on the basis of the 
standard. For example, a classroom test affecting only a few students would 
not require as stringent a procedure as would a statewide test determining 
whether a student is allowed to graduate from high school. Other relevant 
factors include the number of test items, permanence of the standard, purpose 
of the test and the extent of available financial and other resources to 
support the standard-setting process. 
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On February 4, 1985, the Mastery Test Psychometrics Committee met to consider 
the issue of standard-setting procedures and voted unanimously to approve the 
fol lowing proposal . 



A PROPOSAL FOR SETTING THE REMEDIAL STANDARDS ON THE CONNECTICUT MASTERY TESTS 

1. Two standard-setting committees will be created: one for mathematics and 
one for reading and writing. 

2. This description of a minimally proficient student will be given to each of 
the committees: 

Imagine a student who is just proficient enough in reading, writing and 
mathematics to successfully participate in his/her regular eighth-grade 
coursework. 

3. a. In mathematics, an adaptation of the Angoff procedure will be used. 

The committee will be provided with each item appearing on one form of the 
mathematics test. The committee will be'given the following directions: 

Consider a group of 100 of these students who are just proficient 
enough to be successful in regular eighth-grade coursework. How many 
of them would be expected to correctly answer each of the questions?" 

The committee will rate each item. The committee will then be given the 
opportunity to discuss their rating of each item. Sample pilot data will 
be presented. Committee members will be given the opportunity to adjust 
their item ratings. The item ratings will then be averaged in accordance 
with the Angoff procedure in order to produce a recommended test standard. 

b. In reading, the committee will review and discuss each passage of the 
Degrees of Reading Power (DRP) test. Student performance data will be 
presented. The committee will consider the reading difficulty that should 
be expected of a student at the grade level being tested. The committee 
members will identify the passage that has the appropriate level of reading 
difficulty consistent with the above description of a minimally proficient 
student. 

c. In writing, the committee will read four sample essays. These essays 
will have been prescored holistically (on a scale from 2 to 8) in order to 
rank the quality of the essays. Committee members will classify essays 
into one of three categories: 1) definitely NQI proficient, 

2) borderline and 3) definitely proficient. These classifications will be 
discussed in light of the holistic scores. The committee will then 
classify approximately twenty-five additional essays. The essay ratings 
will be discussed in the same manner as the original four essays. When all 
essays have been discussed, the essays which fell in the borderline 
category will be focused upon to determine the standard. The committee 
will determine where, among the borderline essays, the standard should be 
established. 

4. The standards recommended in step 3 will be presented to the Connecticut 
Student Assessment Advisory Committee (formerly the Mastery Test 
Implementation Advisory Committee) for discussion and action. 
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Connecticut's Strategy 



Several steps were employed to create an acceptable and valid test standard 
for Connecticut tests. Initially, a separate standard-setting committee was 
convened for each test on which standards were to be set. Individuals were 
chosen to serve as members on the committee on the basis of their familiarity 
with the area being assessed and the nature of the examinees. One source of 
such members was the test content committees related to the project. For 
example, members of the Mathematics Advisory Committee were represented on the 
committee setting standards for the mathematics mastery test. 

The actual procedures used to set standards were an adaptation of a method 
proposed by William Angoff (1970), This test question review method required 
members of a standard-setting committee to estimate the probability that a 
question would be correctly answered by examinees who possess no more than the 
minimally acceptable knowledge or skill in the areas being assessed. Standard 
setters then reviewed pilot test data for sample items as further evidence of 
the appropriateness of the judgments being made. The original probability 
estimates assigned to each test question were reviewed and adjustments made by 
the standard setters. The final individual item probabilities were summed to 
yield a suggested test standard for each member of the committee. The 
suggested standards were averaged across members of the committee to produce 
the recommended test standard. 

The recommended test standard was presented to the Connecticut Student 
Assessment Advisory Committee and the State Board of Education, 

In mid-March, Mathematics and Language Arts Standard-Setting Committees met to 
set the remedial standards for the Grade 8 Mastery Test, The following 
information summarized the results of the standard-setting activities 
conducted by CSDE staff: 

I, Mathematics (144-item test) 

Using the procedures previously outlined, the standard setters rated each item 
and considered the pilot data. Committee members discussed items and were 
given the opportunity to adjust their initial ratings. The final ratings were 
averaged to produce a remedial standard. It was recommended that a raw score 
of 79 be the remedial mathematics standard. Below is a summary of the ratings. 

Procedure # Judges Range t Mean t Correct Raw Score 

Angoff 20 25,7-67,7 54 78 

II, Reading (Degrees of Reading Power, 77-item test) 

Standard setters used two procedures to establish a remedial reading 
standard. First, they examined the passages in the Degrees of Reading Power 
(DRP) test, asking themselves which passage is too difficult for the student 
who is just proficient enough to successfully participate in eighth-grade 
coursework. Discussion occurred throughout this selection process. 
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Second, they examined textbooks which are typically used in grades seven and 
eight and selected those textbooks which a minimally proficient student would 
not be expected to read in order to successfully participate in eighth-grade 
coursework. Discussion occurred throughout this selection process. 

The average readability values of the selected passages and textbooks and the 
pilot test data were then revealed to the standard setters. The standard 
setters discussed the readability values and the pilot test data and 
recommended the DRP unit score of 55 as the remedial standard. The standard 
was accepted by the State Board of Education at the 80^ comprehension level. 
Below is a summary of the ratings. 



Procedure # Judges 

A. Test Passage Review 26 

B. Textbook Review 26 



Readability 
Range 

53-62 DRP Units 

48-60 DRP Units 



Recommended 
Remedial Standard 



55 DRP Units 



III. Writing (45-minute writing sample) 

Using the procedure previously outlined, standard setters read and rated 21 
essays written to a persuasive prompt and 21 essays written to an expository 
prompt. After discussions and final ratings, the holistic scores for the 
papers were revealed to the group. The committee then discussed the 
appropriate remedial writing standard in light of the degree to which their 
ratings matched the holistic scores. It was the recommendation of the 
committee that a holistic writing score of 4 be used as the remedial writing 
standard. Below is a summary of the ratings. 

PERSUASIVE PROMPT 
Rating After Discussion 



Holistic 
Score 
2 
3 
4 
5 
6 
7 
8 



Definitely 
NOT Proficien t 
100^ 
69% 
271 
01 
61 
M 
01 



Borderl ine 
0^ 
OX 
IX 

ox 
ox 
ox 
ox 



Definitely 
Profici ent 
OX 
31% 
72X 
lOOX 
94X 
99X 
lOOX 



EXPOSITORY PROMPT 

• __ _ A r\. t - • 



Rating After Discussion 



Holistic 
Score 
2 
3 
4 
5 
6 
7 
8 



Definitely 
NOT Proficient 
lOOX 
99X 
17X 
22X 
OX 
OX 
OX 



Borderl ine 
OX 
OX 
IX 

ox 
ox 
ox 
ox 



Definitely 
Proficient 
OX 
IX 
82X 
78X 
lOOX 
lOOX 
lOOX 
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Standard-^Setting Committees 



LANGUAGE ARTS STANDARD-SETTING COMMITTEE 

Dell Britt, Newtown Public Schools 
Fred Brucoli, New London Public Schools 
Patricia Dobson, Stafford Public Schools 
Donald Falcetti, Litchfield Public Schools 
Bill Farr, Bolton Public Schools 
James Foley, Waterbury Public Schools 
Dorothy French, Litchfield Public Schools 
Marguerite Fuller, Bridgeport Public Schools 
Sara Godek, Stafford Public Schools 
Nina Grecenko, Newtown Public Schools 
Mary Haylon, Hartford Public Schools 
Karen Karcheski , Danbury Public Schools 
Robert Kinder, CT State Department of Education 
Jean Klein, Newtown Public Schools 
Mark Kristoff, New London Public Schools 
Thomas Lane, Old Saybrook Public Schools 
Lucretia Leeves, Hartford Public Schools 
Edward Moore, Danbury Public Schools 
Mary Murray, Putnam Public Schools 
Dick Nelson, Old Saybrook Public Schools 
Olive S. Niles, East Hartford Public Schools 
Anne L. Rash, Bolton Public Schools 
Bernice Wagge, Waterbury Public Schools 
Mary Wei nl and, CT State Department of Education 
Mary Wilson, Hartford Public Schools 
Barbara Zamagni, Putnam Public Schools 



MATHEMATICS STANDARD-^SETTING COMMITTEE 

Barbara Bailey, New Haven Public Schools 
Pat Banning, Windham Public Schools 
George Caouette, Manchester Public Schools 
Pearl Caouette, Manchester Public Schools 
Betsy Carter, CT State Department of Education 
Tony Ditrio, Norwalk Public Schools 
Don Flis, West Hartford Public Schools 
Marian Frascino^ Norwalk Public Schools 
Charles Framularo, Bridgeport Public Schools 
Sheryl Hershonick, New Haven Public Schools 
Steve Leinwand, CT State Department of Education 
Mable McCarthy, Middletown Public Schools 
Michele Nahas, Windham Public Schools 
Judy Narveson, Farmington Public Schools 
Mary Ann Papa, West Hartford Public Schools 
Jim Pinto, Bloomfield Public Schools 
Helen Prescott, Ashford Public Schools 
Dolores Vecchiarelli , Westport Public Schools 
Sylvia Webb, Middletown Public Schools 
Frank Whittaker, Bridgeport Public Schools 



APPENDIX E 

Grade Eight Overview of Holistic Scoring 
and 

Marker Papers for Holistic Scoring 



An Overview of Holistic Scoring 



Description of the Method 

Holistic scoring involves judging a writing sample for its total effect. 
The scorer makes an overall evaluation taking into account all characteristics 
which distinguish good writing. No one feature (such as spelling, rhetoric, 
or organization) should be weighted to the exclusion of all other features. 
Contributing to the rationale underlying holistic scoring is evidence that: 

0 no aspect of writing can be judged independently and result in an 
overall score of quality; 

0 teachers can recognize and concur upon good writing samples; and 

0 teachers tend to rank entire pieces of writing in the same way, 
regardless of the importance they might attach to the particular 
components of writing. 

The scoring scale for holistic scoring is determined by the quality of the 
specific samples being evaluated. That is, the success of a particular 
response is determined in relationship to the range of ability reflected in 
the set of writing samples being assessed. 

Preparation for Scoring 

Prior to the training/scoring sessions, a committee consisting of Connecticut 
State Department of Education (CSDE) consultants, representatives of the 
Language Arts Advisory Committee and other language arts specialists from 
throughout the state, two chief readers and a project director from 
Measurement Inc. of Durham, North Carolina, and a reading specialist from The 
Psychological Corporation met and read a substantial number of essays drawn 
from the total pool of essays to be scored. Approximately 60 essays were 
selected to serve as "range-finders" or "marker papers" representing the range 
of achievement demonstrated in the total set of papers. Copies of those 
range-finders served as training papers during the scoring workshops which 
followed. Each range-finder paper was assigned a score according to a 
four-point scale, where 1 represented a poor paper and 4 represented a 
superior paper. 

Scoring Workshops 

During the month of November, several holistic scoring workshops were held in 
various locations throughout the state. Attendance at the grade eight scoring 
workshops totaled 253 teachers. A chief reader and two assistants were 
present at every workshop in addition to representatives of the CSDE. Each 
workshop consisted of a training session and a scoring session. 

Training and Qualifying 

0 All teachers were shown approximately fourteen range-finder papers. 
The chief reader discussed each paper and explained the reason why 
each received its score. 
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0 All teachers were given a six-paper practice set- They scored the 
papers independently and recorded the scores on their papers. When 
all teachers were finished, the chief reader discussed each paper and 
explained why each received its score. 

0 All teachers were given a nine-paper training set. They scored the 
papers independently, based on an overall impression, and recorded 
their scores on a monitor sheet as well as on their papers. As they 
finished reading and scoring, they brought the monitor sheet to the 
team leader who checked the scores. When all teachers were finished 
and all monitor sheets were checked, the chief reader discussed the 
nine-paper set, 

0 Regardless of whether or not they qualified on the first training 
set, all teachers were then given another nine-paper training set. 
They scored the papers and had the monitor sheets checked. Set Two 
was not discussed, except with non-qualifiers, 

0 Teachers were considered qualified if they scored six or more papers 
correctly on either set. Teachers who met the standard began scoring 
actual test papers after Set Two, 

0 If any teacher did not qualify, they received additional training by 
one of the team leaders or by the chief reader away from the scoring 
room. They had two more opportunities to qualify. Any teacher who 
failed to qualify would have been excused from the project and paid 
for one day. 

The Scoring Session 

Once scorers qualified, actual scoring of the writing exercises began 
according to the steps outlined below: 

o Scorers read each paper once carefully but quickly and designated a 
score. Again, the score reflected the scorer's overall impression of 
the response as it corresponded with the features of written 
composition which were internalized during the training process. 

0 Each paper was read and scored by a second scorer independently of 
the first, that is, without seeing the score assigned by the first 
reader. The chief reader had the responsibility of adjudicating any 
disagreement of more than one point between the judgments of the 
first two scorers. In other words, adjacent scores (i.e., awarded 
scores of 4 and 3, 1 and 2, 2 and 3) were acceptable, but larger 
discrepancies (i.e., scores of 2 and 4, 3 and 1, 1 and 4) were 
resolved by the chief reader. In general, with successful training, 
the occurrence of large score discrepancies is rare. 

0 The two scores for each paper were added to produce the final score 
for each student, resulting in scores between 2 and 8, 



-51- 8G 



Understanding the Holistic Scores 



Examples of actual student papers which are representative of the scoring 
range will assist the reader in understanding the statewide standard set for 
writing and interpreting the test results. Sample papers representing four 
different holistic scores are presented on the following pages. Note that the 
process of summing the scores assigned by the two readers expands the scoring 
scale to account for "borderline" papers. A paper which receives a 4 from 
both scorers (for a total score of 8) is likely to be better than a paper to 
which one reader assigns a 4 and another reader assigns a 3 (for a total score 
of 7). In addition, it should be emphasized that each of the score points 
represents a range of student papers — some 4 papers are better than others. 

A score of Not Scorable (NS) was assigned to student papers in certain cases. 
A score of NS indicates that the student's writing skills remain to be 
assessed. The cases in which a score of NS was assigned were as follows: 

0 responses merely repeated the assignment; 

0 illegible responses; 

0 responses in languages other than English; 

0 responses that failed to address the assigned topic in any way; and 

0 responses that were too brief to score accurately, but which 

demonstrated no signs of serious writing problems (for example, a 
response by a student who wrote the essay first on scratch paper and 
who failed to get very much of it copied). 

Both readers had to agree that a paper deserved a NS before this score was 
assigned. If the two readers disagreed, the Chief Reader arbitrated the 
discrepancy. Papers which were assigned a score of NS were not included in 
summary reports of test results. 

Summary Comments 

The fact that standards must be maintained and reinforced throughout a scoring 
session cannot be overemphasized. Holistic scoring depends for its usefulness 
on consistency of scoring among all scorers throughout the sessions. 
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CONNECTICUT MASTERY TEST 
1991 Grade Eight 
Writing Assignment 



A 16-year-old friend is thinking about dropping out of school. Should people 
this age quit school? You want to persuade your friend to have the same 
opinion about quitting school that you have. How would you persuade your 
friend? 

• Think about all the reasons for and against dropping out of school. 

• Decide whether you think dropping out of school is good for 
16-year-olds. 

• Write an essay to persuade your friend to agree with your decision. 
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APPENDIX F 
Grade Eight Analytic Rating Guide 
and 

Marker Papers for Analytic Scoring 



Grade Eight Analytic Rating Guide 
FOCUS: How effectively does the writer unify the paper by a dominant topic? 

1 m switches and/or drifts frequently from the dominant topic 

2 « switches and/or drifts somewhat from the dominant topic 

3 « stays on topic throughout the response 

ORGANIZATION: Is there a plan that clearly governs the sequence from the 
beginning to the end of the response, and is the plan effectively signaled? 

1 « no discernible plan 

2 « inferable plan and/or discernible sequence; some signals may be 

present 

3 - controlled, logical sequence with a clear plan 

SUPPORT/ELABORATION: To what extent is the narrative developed by details 
that describe and explain the narrative elements (character, action and 
setting)? 

1 - vague or sketchy details that add little to the clarity of the 

response or specific details but too few to be called list-like 

2 - details that are clear and specific but are list-like, or uneven, or 

not developed 

3 - somewhat developed details that enhance the clarity of the response 

CONVENTIONS: To what extent does the student use the conventions of 
standard written English (e,g*, sentence formation, spelling, usage, 
capitalization, punctuation)? 

1 - many errors 

2 m some errors 

3 « few errors 
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APPENDIX G 

Sample Grade Eight Mastery Test Score Reports 

0 Class Diagnostic Report 

- Mathematics 

0 School by Class Report 

- Mathematics 

0 District by School Report 

- Mathematics 

0 Class Diagnostic Report 

- Language Arts 

0 School by Class Report 

- Language Arts 

0 District by School Report 

- Language Arts 

0 Parent/Student Diagnostic Report 
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Fall 1991 Grade Eight 
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Type of Community 



TOC 1 - LARGE CITY - a town with a population of more than 100,000. 

TOC 2 » FRINGE CITY - a town contiguous with a large city, and with a 
population over 10,000. 

TOC 3 - MEDIUM CITY - a town with a population between 25,000 and 100,000 and 
not a Fringe City. 

TOC 4 - SMALL TOWN (Suburban) - a town within an SMSA* with a population of 
less than 25,000, not a Fringe City. 

TOC 5 - SMALL TOWN (Emerging Suburban) - a town with a population of less than 
25,000 included in what was a proposed 1980 SMSA but not included in a 
1970 SMSA. 

TOC 5 - SMALL TOWN C^'ral) - a town not included in an SMSA, with a population 
of less than 25,000. 



*Standard Metropolitan Statistical Area 
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Education Reference Group Descriptions 



The education reference groups were formed from an analysis of districts' 
median family income, a percentage of high school graduates, a percentage of 
those in managerial/professional occupations, a percentage of single-parent 
families, a percentage of those below poverty and a percentage of non-^English 
home language from the 1980 census. The groups have not been named, but have 
been labeled I through VII. Note, however, that the groups run from extremely 
affluent suburban communities (I) to our three largest cities of Hartford, 
Bridgeport and New Haven (VII). Some differ widely with respect to all of the 
family background variables; others differ slightly with respect to one or 
two. In addition to the six variables used to classify districts, the group 
descriptions below also include superintendents* comments that were provided 
in a Department survey in 1988. 

Group I. These 13 districts were wealthy, professional suburbs. The median 
family income in 1979 averaged $40,425. Residents were extremely well 
educated. Nearly 90% had at least a high school diploma, 42X had a bachelor's 
degree and 49% had a managerial or professional job. There were relatively 
few children with educational disadvantages here. Only 7% of the families 
were single-parent, about 8% spoke a language other than English at home and 
almost no one (2X) lived in poverty. Superintendents within these towns used 
the adjectives "suburban," "affluent," "growing" and "bedroom community" to 
describe them. 

Group II. Residents in the 29 districts of Group II were affluent, 
well-educated professionals, but to a lesser extent than residents of 
Group I. The median family income averaged $28,113, more than 83X of the 
residents had high school diplomas, 29% had a college degree and 36% had a 
managerial or professional job. Like Group I, this group had a low percentage 
of people who spoke another language at home (8%), almost no one in poverty 
(2X) and relatively few single-parent families (9%). Like the superintendents 
in Group I, superintendents from these towns described their communities as 
"affluent," "bedroom communities," "growing" and "suburban." 

Group III. These 34 districts were mostly rural bedroom communities. Like 
Groups I and II, these towns did not have many disadvantaged children. There 
were only 7X who spoke a language other than English at home, only 7% who were 
from single-parent families and only 3% who were poor. Adults were slightly 
less affluent (median family income of $24,431), less likely to have a high 
school diploma (77%) and less likely to have a managerial or professional job 
(28X) than people in Group II. Like the previous two groups, these towns were 
described by superintendents as "suburban," "growing" and "bedroom 
communities." Several superintendents used "rural" and "middle class" (as 
well as "affluent") to describe their communities. 
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Group IV, This group of 37 districts was probably the most diverse set of 
towns, containing a number of coastal and resort communities, as well as rural 
and suburban areas. Group IV was similiar to Group III in median family 
income ($22,609), percentage of high school graduates percentage of 

managers/professionals (29%) and percentage of non-English home language (7%). 
but had a significantly higher percentage of single-parent families 
(12X versus 7X) and a slightly higher percentage of families below poverty 
(5X versus 3X), Superintendents' descriptions reflect this group*s 
diversity. They describe their towns as "bedroom." "growing." "rural." 
"suburban." "middle income" and "affluent," 

Group V. These 30 districts made up the first group of working class/blue 
collar communities. This group had a significantly lower percentage of high 
school graduates (68X) and percentage of managers/professionals (19%) than 
Group IV. Other characterisitics were similar to Group IV: the average income 
was $21,920. there were IIX single-parent families. 51 below poverty and 9% of 
the population spoke a language other than English at home. 

Group VI, This group of 23 districts included the state's medium-sized 
cities, the larger cities of Stamford and Waterbury, several former mill towns 
and some densely populated blue collar suburbs. Group VI had similar 
socioeconomic characteristics as Group V. but significantly greater 
proportions of single-parent families and families in which English was not 
the primary home language. The median family income of $20,325 was below the 
state average. An average of 16X of the residents spoke another language at 
home, and 17X of the families were heaoed by single parents. Only 63X of the 
residents had high school diplomas, and 6X lived below poverty level. 

Group VII. Hartford, Bridgeport and New Haven were vastly different from 
other communities in Connecticut, An average of 28X of the families spoke a 
language other than English, 46X were headed by single parents. 20X lived in 
poverty and the median family income was $15,240. 
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Connecticut State 
Department of Education 



Division of Research, Evaluation and Student Assessment 

Douglas A. Rindone, Acting Director 

Bureau of Evaluation and Student Assessment 

Peter Behuniak, Acting Chief 

Student Assessment and Testing Unit 

William J. Congero, Coordinator 

Patricia Brandt 
Richard Cloud 
Naomi Wise 
Education Service Specialists 

John B. Rogers 
Education Service Assistant 

Richard F. Mooney, Senior Program Associate 

Stephen Martin, Assessment Associate 

Division of Curriculum and 
Professional Development 

Betty Sternberg, Director 

Steven Leinwand 
Mari Muri 
Mathematics Consultants 

Karen Costello 
Reading/Language Arts Consultant 



It is the policy of the Connecticut State Board of Education that no person shall be excluded 
from participation in, denied the benefits of, or otherwise discriminated against under any 
program, including employment, because of race, color, religion, sex, age, national origin, 
ancestry, marital status, mental retardation, mental or physical disability, including, but not 
limited to blindness. 
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